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after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
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DETAILED ACTION 

1. The amendment filed on 10 December 2004 have been received and entered. The 
applicant has cancelled claims 6, 16, 25 and 34. Claims 1-5, 7-15, 17-24, 26-33 and 35-38 as 
amended are pending in the application. 



Claim Rejections - 35 USC § 103 
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

2. Claims 1-5, 7-15, 17-24, 26-33 and 35-38 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over the article entitled "Color SuperHistograms for Video Representation", written 
by Dimitrova et al., and Wang et al. U.S. Patent 5,805,733. 

Referring to claims 1, 11,21 and 30, Dimitrova et al. teach an apparatus, system, method 
and computer executable instructions comprising a visual summary controller capable of creating 
a visual summary of video material (Dimitrova et al.: page 316, Figure 1), wherein the visual 
summary controller is capable of extracting frame signatures (histograms) from keyframes of 
video material and capable of using the frame signatures to create superhistograms from the 
keyframes (Dimitrova et al.: page 314, right column, lines 1 1-25, page 315, section 2 and page 
316, section 2.3; this is further shown in Figure 1). However, although Dimitrova et al. teach 
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using the frame signatures and superhistograms to create a visual summary of video material in a 
broad sense (representing video segments by computing superhistograms) (Dimitrova et al.: 
Abstract), Dimitrova et al. fail to explicitly teach selecting representative keyframe images for 
each superhistogram to create a compact visual summary of the video material, wherein the 
representative images include at least one of the first image in each family histogram, the most 
meaningful image in each superhistogram, a randomly chosen image and an image that is closest 
to the cluster center. Wang et al. teach the analysis of scenes and frames in video materials 
(Wang et al: column 1, lines 53-56 and Figure 2) similar to that of Dimitrova et al. In addition, 
Wang et al. further teach selecting representative keyframe images from each group of related 
scenes to create a compact visual summary of the video material (summarizing a video sequence 
by taking one representative frame from each set of related scenes with similar average color 
histograms, to represent the set to enable the user to view a large sampling of video sequence 
images) (Wang et al.: column 1, lines 51-67 and column 2, lines 1-24; this is further shown in 
Figure 3), wherein the representative images include at least one of the first image in each family 
histogram, the most meaningful image in each superhistogram, a randomly chosen image and an 
image that is closest to the cluster center (the representative frame image can be taken from the 
temporally medial scene in the set or from one of the frames of the longest scene in the set of 
related scenes) (Wang et al.: column 3, lines 37-66). It would have been obvious to one of 
ordinary skill in the art, having the teachings of Dimitrova et al. and Wang et al. before him at 
the time the invention was made, to modify the visual summary controller capable of extracting 
frame signatures from keyframes to create superhistograms of Dimitrova et al., to include the 
further step of selecting representative keyframes from those superhistograms and using the 
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representative keyframe images to create a compact visual summary, taught by Wang et al. One 
would have been motivated to make such a combination in order to meet the need of being able 
to readily access and manipulate video information, by cataloguing and storing the potentially 
thousands of hours of video for rapid future retrieval, browsing and use, created by the 
increasing availability and use of digital video and the increasing integration of computer 
technologies and video production technologies. 

Referring to claims 2, 12, 22 and 31, Dimitrova et al. teach the filtering of keyframes 
(merging of histograms into family histograms) and extracting frames signatures (computing 
color histograms) from the filtered keyframes before using the frame signatures (histograms) to 
create the superhistogram representing a visual summary of the video material (page 315, right 
column, section 2 and page 316, left column, section 2.3). 

Referring to claims 3, 13, 23 and 32, Dimitrova et al. teach the use of superhistograms to 
cluster the filtered keyframes (the ordered merging of the family histograms to create the 
superhistogram), wherein the clustered keyframes (superhistogram) represents the visual 
summary of the video material, as recited on page 314, right column, lines 1 1-25 and shown in 
Figure 1. 

Referring to claims 4 and 14, Dimitrova et al. teach the use of a histogram as the frame 
signature used to compute superhistograms (page 314, right column, lines 11-15). 

Referring to claims 5, 15, 24 and 33, Dimitrova et al. the use of the LI distance measure 
method, L2 distance measure method, histogram intersection method, Chi-Square test and Bin- 
wise histogram intersection method to computer the histogram difference (page 315, right 
column). 
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Referring to claims 7, 17, 26 and 35, Dimitrova et al. teach the ability to select the family 
histograms (the top n largest families) to use to create the superhistogram used to create the 
visual summary (page 316, section 2.4). 

Referring to claims 8, 18, 27 and 36, while Dimitrova et al. teach all of the limitations as 
applied to claims 1, 11,21 and 30 above, Dimitrova et al. fail to explicitly teach the capability to 
retrieve a visual summary stored in a memory unit and causing the visual summary to be 
displayed in response to a user request. Wang et al. teach the analysis of scenes and frames in 
video materials (Wang et al.: column 1, lines 53-56 and Figure 2) similar to that of Dimitrova et 
al. In addition, Wang et al. further teach the capability of letting a user select a visual summary 
for viewing, retrieving that visual summary from memory and displaying it in response to the 
user's request (displaying visual summaries of scenes in a movie bar and allowing users to 
access the summaries by selecting the segments corresponding to the scenes) (Wang et al.: 
column 2, lines 16-29 and shown in Figures 2 and 3). It would have been obvious to one of 
ordinary skill in the art, having the teachings of Dimitrova et al. and Wang et al. before him at 
the time the invention was made, to modify the visual summary controller capable of extracting 
frame signatures from keyframes to create superhistograms of Dimitrova et al., to include the 
retrieval and display of the visual summary in response to a user request, as taught by Wang et 
al. One would have been motivated to make such a combination to give users the flexibility to 
select which scenes to watch, saving them time from having to browse through all of the other 
irrelevant scenes; furthermore, because the increasing availability and use of digital video and 
the increasing integration of computer technologies and video production technologies have 
produced the need to be able to readily access and manipulate video information, it would have 
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been advantageous to make such a combination in order to provide users a way to summarize the 
content of video quickly and easily, in order to catalogue and store the potentially thousands of 
hours of video for rapid future retrieval, browsing and use. 

Referring to claims 9, 19, 28 and 37, Dimitrova et al. teach the use of the visual summary 
obtained from the superhistograms to access at least a portion of the video material (classifying 
and searching in video archives), as recited on page 317, section 4.2. 

Referring to claim 10, 20, 29 and 38, while Dimitrova et al. teach all of the limitations as 
applied to claims 1, 1 1, 21 and 30 above, Dimitrova et al. fail to explicitly teach the creation of 
new video material using the compact visual summaries. Wang et al. teach the analysis of scenes 
and frames in video materials (Wang et al.: column 1, lines 53-56 and Figure 2) similar to that of 
Dimitrova et al. In addition, Wang et al. further teach the creation of new video material using 
the compact visual summaries (a collage made up of representative frames for each set of 
summarized scenes) (Wang et al.: column 3, lines 53-57). It would have been obvious to one of 
ordinary skill in the art, having the teachings of Dimitrova et al. and Wang et al. before him at 
the time the invention was made, to modify the visual summary controller capable of extracting 
frame signatures from keyframes to create superhistograms of Dimitrova et al., to include the 
creation of new video material, as taught by Wang et al. It would have been advantageous for 
one to utilize such a combination in order to conserve processor time and storage space by 
utilizing the already existing visual summaries in the creation of new visual materials; 
furthermore, because the increasing availability and use of digital video and the increasing 
integration of computer technologies and video production technologies have produced the need 
to be able to readily access and manipulate video information, it would have been advantageous 



Application/Control Number: 09/866,394 Page 7 

Art Unit: 2173 

to make such a combination in order to provide users a way to summarize the content of video 
quickly and easily, in order to catalogue and store the potentially thousands of hours of video for 
rapid future retrieval, browsing and use. 

Response to Arguments 

3. Applicant's arguments filed 10 December 2004 have been fully considered but they are 
not persuasive. 

4. The applicant asserts that Wang et al. do not teach or disclose the limitations of claims 6, 
16, 25 and 34, which have been incorporated into independent claims 1,1121 and 30, 
respectively. Specifically, the applicant asserts that Wang et al. do not teach or disclose that 
"said representative images include at least one of (1) the first image in each family histogram, 
(2) the most meaningful image in each superhistogram, (3) a randomly chosen image, and (4) an 
image that is closes to the cluster center". The examiner respectfully disagrees. Wang et al. 
teach selecting a representative frame image for each set of summarized related scenes, as recited 
in column 3, lines 37-57. Furthermore, Wang et al. teach that the representative frame can be 
taken from the temporally medial scene in the set, i.e. the representative frame can be a frame 
that is halfway between the first and last scenes, as recited in column 2, lines 12-15 and column 
3, lines 57-59; in other words, the representative frame can include a frame that is in the middle, 
or center, of the cluster of scenes. In addition, Wang et al. also teach that the representative 
frame can be a frame taken from the longest scene, since the longest scene is most indicative of 
the content of the related scenes, as recited in column 3, lines 59-62; in other words, the 
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representative frame can include a frame that is the most indicative of the contents of the related 
scenes, i.e. the most meaningful frame in the group. Therefore, it can be seen that Wang et al. 
teach that the representative images includes at least one of the first image in each family 
histogram, the most meaningful image in each superhistogram, a randomly chosen image and 
an image that is closest to the cluster center. 

5. Applicant's amendment necessitated the new ground(s) of rejection presented in this 
Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Ting Zhou whose telephone number is (571) 272-4058. The 
examiner can normally be reached on Monday - Friday 8:30 am - 6:00 pm. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, John Cabeca can be reached at (571) 272-4048. The fax phone number for the 
organization where this application or proceeding is assigned is (571) 273-4058. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 

19 January 2005 
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